Repository navigation
feat(skills): give Claude Agents runs the skills attached to their variation - #208
Merged
Merged
Conversation
…host settings A run with no NativeTool attached now sends tools=[], so the Claude Code built-ins (Read, Bash, Skill and the rest) are off unless a config attaches them. 50e82fd omitted the key instead, which gave every run the full built-in set. TypeScript has sent tools=[] since its first release. TESTING.md §2.x.2. create_claude_agents_handler takes cwd and setting_sources, forwarded to every query() on both paths. setting_sources not given sends [], because the Agent SDK reads None as "load every source": the host's plugins, skills, hooks, permission rules and CLAUDE.md reached every run (AIC-3584). cwd is sent only when given. to_claude_agents also sends setting_sources=[]. §2.x.1. BREAKING CHANGE: runs no longer get Claude Code built-ins unless the config attaches them as NativeTool sentinels, and no longer load host Claude Code settings unless setting_sources names them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…riation
A skill attached to an agent-mode variation never reached a model run
through create_claude_agents_handler: tools=[] removes the Skill tool,
and nothing hands the CLI the skill files.
create_claude_agents_skills_handler(), from the new
launchdarkly_ai_claude_agents.experimental.skills module, does. On every
invoke and stream it:
- writes skill_refs(config) with write_skills, through the store
configured with set_skill_store, into
mkdtemp(<skills_temp_dir>/ld-skills-)/launchdarkly/skills;
- passes plugins=[{"type": "local", "path": <dir>/launchdarkly}] and
skills=["launchdarkly:<key>", ...], and appends Skill to tools (not to
allowed_tools);
- removes the directory when the run ends, however it ends.
It fails closed: a reconcile that is not ok raises before query(),
naming each skill. A variation with no skills runs exactly as with the
core handler and needs no store. setting_sources is always sent, so
claude-agent-sdk never falls back to user,project once skills is set.
Core reaches this only through a private _prepare keyword on the
internal factory. The package root does not import the experimental
module.
Spec: TESTING.md §2.x Claude Agents Skills Integration and A.18
(launchdarkly/ai-sdks-monorepo#59).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
XieX
marked this pull request as ready for review
October 23, 2026 17:36
…t-options # Conflicts: # packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py
…native adapter Matches launchdarkly/js-ai-sdk#135. claude_agents(), claude_graph() and to_claude_agents() hard-coded setting_sources=[], so a wrapper user could not opt back in to host settings, and these signatures freeze at 1.0. - claude_agents and claude_graph take cwd / setting_sources keyword-only, build the handler with them, and pass neither on to config() / graph(). - to_claude_agents reads both from opts and sends _host_query_options(...) to every node's query(), root and sub-agents, in place of the hard-coded []. - The README gains a migration note: [] also drops the settings-file env (ANTHROPIC_BASE_URL, CLAUDE_CODE_USE_BEDROCK / _VERTEX, proxies), apiKeyHelper and model, so a Bedrock, Vertex or gateway deployment configured through ~/.claude/settings.json stops authenticating. - agents.md §2a describes the model-parameter allowlist from main instead of "forwards none of model.parameters". - API report regenerated for the new signatures. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kly/python-ai-sdk into xie/claude-agents-skills # Conflicts: # packages/claude-agents/src/launchdarkly_ai_claude_agents/handler.py
…xperimental.skills Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…spans raises The blocking path ran cleanup after finish_root_span and end_unfinished_spans in the same finally, so a raise from either leaked the run's plugin directory. Cleanup now has its own finally. Lists the experimental skills module in FORWARDED_LISTS: it delegates to the core handler, which forwards the parameters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
This needs a human reviewer. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 85d37ee. Configure here.
…stream spans raises The stream path closed the vendor generator and ran the preparation's cleanup after inference.finish(), abandon_tool_spans and end_span_once in the same finally, so a raise from any of them skipped both and leaked the run's plugin directory. They now run in their own finally, as on the blocking path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jeffdupont
approved these changes
Oct 23, 2026
jeffdupont
added a commit
that referenced
this pull request
Oct 26, 2026
#208 added the Claude Agents skills handler while Agent Skills was still experimental, so it imported launchdarkly_ai_server.experimental.skills. This PR removes that module, which broke the handler, its tests and mypy once main was merged in. The handler itself stays in launchdarkly_ai_claude_agents.experimental.skills. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
XieX
added a commit
that referenced
this pull request
Oct 26, 2026
…e root The handler is promoted with Agent Skills (#210), so it moves out of launchdarkly_ai_claude_agents.experimental.skills before its first release (#208 is unreleased): - create_claude_agents_skills_handler is exported from launchdarkly_ai_claude_agents, from a new skills module. The experimental subpackage and its API report are gone. - It reports claude-agents.createClaudeAgentsSkillsHandler. Spec: launchdarkly/ai-sdks-monorepo#59. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
XieX
added a commit
that referenced
this pull request
Oct 26, 2026
…e root (#217) #208 added `create_claude_agents_skills_handler` at `launchdarkly_ai_claude_agents.experimental.skills`. The spec now promotes it with Agent Skills (launchdarkly/ai-sdks-monorepo#59, following #86), and no release has shipped it, so it moves to the package root with no deprecated alias: - `create_claude_agents_skills_handler` is exported from `launchdarkly_ai_claude_agents`, from a new `skills` module (moved from `experimental/skills.py`). The `experimental` subpackage and its API report are deleted. - It reports `claude-agents.createClaudeAgentsSkillsHandler` instead of `experimental.createClaudeAgentsSkillsHandler`. - The README section loses its experimental notice, and the example imports from the package root. agents.md follows. - Tests: the fresh-interpreter test that the package root never loaded the experimental module is replaced by one that the root exports the handler. The usage test, and `tests/test_never_forwarded_parameters.py`'s module map, use the new module and ID. **Phantom usage events (review follow-up).** The handler called the public `skill_refs` and `write_skills`, so every run with skills also reported `client.skillRefs` and `client.writeSkills`, calls the customer never made. This was already on main from #208/#210; this PR makes the handler a core export, so it fixes it here: - Core gains a non-reporting `_skill_refs` in `launchdarkly_ai_server.skills`; `skill_refs` reports and then delegates to it. Not a public name, so no API report changes. - The handler uses `_skill_refs` and the existing `_write_skills` from `launchdarkly_ai_server.skills_fs`, as `skills_watch.py` already does. - `packages/claude-agents/tests/test_sdk_usage.py` gains a run with skills, over invoke and stream, that sends only `claude-agents.createClaudeAgentsSkillsHandler`. The test fails against the public helpers. The TypeScript twin is launchdarkly/js-ai-sdk#136, which makes the same move before it merges and likely has the same usage pattern to fix. **Tests:** rebased onto `main` after #210 squash-merged, so the branch carries only its own commits. `make api-report-check` reports every report current. `make lint`, `make format-check` and `make typecheck` pass, and the full suite passes locally. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > **`create_claude_agents_skills_handler`** graduates from `launchdarkly_ai_claude_agents.experimental.skills` to a top-level **`skills`** module and is re-exported from **`launchdarkly_ai_claude_agents`**. The experimental subpackage and its API report are removed; docs and examples import from the package root, and SDK usage is reported as **`claude-agents.createClaudeAgentsSkillsHandler`** instead of the experimental helper id. > > Runs that materialize variation skills no longer emit phantom **`client.skillRefs`** / **`client.writeSkills`** events. The handler calls internal **`_skill_refs`** and **`_write_skills`** (core adds **`_skill_refs`** as the non-reporting body behind public **`skill_refs`**). New usage tests assert invoke/stream with skills only records the factory helper. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit d288473. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
jeffdupont
added a commit
to launchdarkly/js-ai-sdk
that referenced
this pull request
Oct 26, 2026
…riation (#136) Base is `main`, which now includes #135 (host options) and #159 (Agent Skills promotion). **This handler is promoted with Agent Skills**, so it ships from the package root and never from `/experimental` (spec: launchdarkly/ai-sdks-monorepo#59). A skill attached to an agent-mode variation never reached a model run through `createClaudeAgentsHandler`. Measured on Agent SDK 0.3.220 by reading the CLI's `init` message: skills are *discovered*, but the handler's `tools: []` removes the `Skill` tool, and no `query({ skills })` value brings it back. `createClaudeAgentsSkillsHandler(options)`, exported from `@launchdarkly/ai-claude-agents`, fixes that. On every invoke and stream it: 1. Writes `skillRefs(config)` with `writeSkills` (both from `@launchdarkly/ai-server`), through the store configured with `setSkillStore`, into `mkdtemp(<skillsTempDir>/ld-skills-)/launchdarkly/skills`. Pinned versions are honored. 2. Passes `plugins: [{ type: 'local', path: <dir>/launchdarkly, skipMcpDiscovery: true }]` and `skills: ['launchdarkly:<key>', …]`, and appends `Skill` to `tools` (not to `allowedTools`). 3. Removes the directory when the run ends, however it ends. - **One directory per run.** There's no shared root, no watcher and no prune. Detaching a skill from a variation takes effect on the next run, and concurrent runs never share files. - **Fail closed.** If the reconcile isn't `ok` (no store, store not yet initialized, a skill withheld by verification, or an absent pin), the run throws before `query()` and names each skill. - **No-skills path.** A variation with no skills runs exactly as with the core handler, and needs no store. - **Telemetry.** The tool hooks are installed whenever `Skill` is in `tools`, so a `Skill` call gets an `execute_tool Skill` span. - **Internal hook.** The handler reaches core through a package-internal `prepare` hook on `createClaudeAgentsHandlerInternal`, which isn't part of `createClaudeAgentsHandler`'s public options. - **Usage ID.** It reports `claude-agents.createClaudeAgentsSkillsHandler`, not the core factory's ID. - **Options.** `ClaudeAgentsSkillsHandlerOptions` is `ContentCaptureOptions & ClaudeAgentsHostOptions & { skillsTempDir? }`. `ClaudeAgentsHostOptions` is now exported from the root too. - **Not covered:** skills through `claudeAgents()`, `claudeGraph()` and `toClaudeAgents()`. They take the host options (#135) but build the core handler, so a variation run through them doesn't get its skills. The README says so. **Tests:** a new `skills.test.ts` with a mocked `query()` and a real store, filesystem and OTel provider, plus the usage row in `sdk-usage.test.ts`. `api-surface/ai-claude-agents.txt` lists the new names marked `+`. Lint, typecheck, `api:update` / `api:surface` and all 216 claude-agents tests pass locally. **Verified live:** a model loads the skill with `Skill` in `tools` but not in `allowedTools`, on invoke and stream, against 11c6cb4. See [the live-run comment](#136 (comment)). The earlier spike also showed that the handler returns the CLI's "Credit balance is too low" result as a successful output; that's tracked separately. **Spec:** TESTING.md §2.x "Claude Agents Skills Integration" and A.21, in launchdarkly/ai-sdks-monorepo#59. **Python:** launchdarkly/python-ai-sdk#208 merged it at `experimental.skills`, unreleased; a follow-up PR moves it to the root. 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Overview** > Adds **`createClaudeAgentsSkillsHandler`** so agent-mode Claude runs can use LaunchDarkly variation **skills**: on each invoke/stream it writes pinned skill refs from `setSkillStore` into a per-run temp plugin (`launchdarkly:<key>`), enables the Agent SDK **`Skill`** tool via a local plugin, and deletes the directory in **`finally`** (including after stream abandonment or span-processor errors). > > The core handler gains a package-internal **`prepare`** hook on `createClaudeAgentsHandlerInternal` (`QueryPreparation`: extra `query()` options, **`extraBuiltins`** on `tools`, cleanup). **`@launchdarkly/ai-server`** exposes **`skillRefsInternal`** / **`writeSkillsInternal`** so handler plumbing does not double-count SDK usage. Variations with no skills behave like **`createClaudeAgentsHandler`**; skill write/verify failures **fail closed** before `query()`. **`claudeAgents`**, **`claudeGraph`**, and **`toClaudeAgents`** are unchanged. > > Also exports **`ClaudeAgentsHostOptions`** / **`ClaudeAgentsSkillsHandlerOptions`**, README/agents docs, API-surface entries, and a large **`skills.test.ts`** suite. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 293f11b. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY --> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Jeff Dupont <jdupont@launchdarkly.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Stacked on #201 (host options and built-in tool set). Review that one first. This PR's base is its branch.
A skill attached to an agent-mode variation never reached a model run through
create_claude_agents_handler. #201'stools=[]removes theSkilltool, and nothing hands the CLI the skill files.create_claude_agents_skills_handler(*, capture_content=None, cwd=None, setting_sources=None, skills_temp_dir=None), from the newlaunchdarkly_ai_claude_agents.experimental.skillsmodule, fixes that. On every invoke and stream it:skill_refs(config)withwrite_skills, through the store configured withset_skill_store, intomkdtemp(<skills_temp_dir>/ld-skills-)/launchdarkly/skills. Pinned versions are honored.plugins=[{"type": "local", "path": <dir>/launchdarkly}]andskills=["launchdarkly:<key>", …], and appendsSkilltotools(not toallowed_tools).ok(no store, store not yet initialized, a skill withheld by verification, or an absent pin), the run raises beforequery()and names each skill.setting_sourcesis neverNone. Withskillsset andsetting_sources=None,claude-agent-sdksends--setting-sources=user,project. The core handler already always sends it ([]unless given), and this handler inherits that.Skillis intools, so aSkillcall gets anexecute_tool Skillspan, with no$ld:ai:tool_callevent._preparekeyword argument on_create_claude_agents_handler. It is not an option of the public factory. The package root doesn't import the experimental module, and a test checks that in a fresh interpreter.claude_agents(),claude_graph()andto_claude_agents(). The README says so.Differences from TypeScript
skipMcpDiscovery.claude-agent-sdk'sSdkPluginConfighas no such key, so the plugin entry carries onlytypeandpath. The directory holds no MCP config, so there is nothing to discover.shutil.rmtree, so a cancellation during teardown can't interrupt it. A failed removal is logged and doesn't replace the run's outcome.Tests
test_skills.py(49 tests) uses a mockedquery()with a realInMemorySkillStore, a real filesystem (tmp_pathasskills_temp_dir) and a real OTel provider. Each spec bullet runs on both invoke and stream where it applies.test_sdk_usage.pyadds theexperimental.createClaudeAgentsSkillsHandlerrow.Skillfrom the hook-install condition fails the span tests.Checks:
uv run pytest: everything passes exceptopencode-agentstest_a_fake_opencode_is_started_spoken_to_and_stopped, the macOS/private/vartemp-path failure already onmain.make lint,make format-checkandmake typecheckare clean.Follow-ups in the monorepo
EXPERIMENTAL.mdrow for Python.api-surface/python/launchdarkly_ai_claude_agents.experimental.skills.txt.pythononexperimental.createClaudeAgentsSkillsHandlerinusage-ids.txt.Not verified yet: a live run where a model actually loads the skill.
Spec: TESTING.md §2.x "Claude Agents Skills Integration" and A.18, in launchdarkly/ai-sdks-monorepo#59. The TypeScript side is launchdarkly/js-ai-sdk#136.
🤖 Generated with Claude Code
Note
Overview
Adds experimental
create_claude_agents_skills_handler()underlaunchdarkly_ai_claude_agents.experimental.skillsso Claude Agent runs get the skills attached to their AI Config variation. The corecreate_claude_agents_handler()path is unchanged and still ignores variation skills.On each invoke/stream with skills, the handler writes skill files from
set_skill_storeinto a per-run temp directory, passes it toquery()as a locallaunchdarklyplugin with qualifiedlaunchdarkly:<key>skill names, enables theSkillbuilt-in (via new internal_prepare/QueryPreparationon_create_claude_agents_handler), and removes the directory in a final cleanup on success, failure, cancel, or stream abandonment. Reconcile failures fail closed beforequery(); empty skills behave like the core handler.Docs, API report, usage tracking for
experimental.createClaudeAgentsSkillsHandler, and a largetest_skills.pysuite cover materialization, concurrency, telemetry (execute_tool Skill), and thatclaude_agents/ graph helpers are not wired yet.Reviewed by Cursor Bugbot for commit ac743e0. Bugbot is set up for automated code reviews on this repo. Configure here.